Skip to content

fix(dag): session-following location stamps + adoption-vs-deletion fencing (#269, #270) - #272

Merged
LeXwDeX merged 3 commits into
devfrom
fix/adoption-state-machine
Aug 15, 2026
Merged

fix(dag): session-following location stamps + adoption-vs-deletion fencing (#269, #270)#272
LeXwDeX merged 3 commits into
devfrom
fix/adoption-state-machine

Conversation

@LeXwDeX

@LeXwDeX LeXwDeX commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Why

What

  • SessionProjector re-stamps workflow execution-location from the Moved event payload inside the same durable publish transaction (payload-sourced, no SessionTable read) — the stamp moves with the session (fix(dag): SessionMoved wake wedge — mixed directory stamps leave workflows with no directory owner #269).
  • One conditional UPDATE claim in DagStore (tryClaimAdoption): status whitelist ['pending','running','paused','stepping'] (exactly the non-terminal WorkflowStatus set); no lease column, no fail-open gap. Gated after ownsWorkflow at both adoption sites and at spawn re-admission (fix(dag): vs-deletion adoption tail — adoption can slip past a workflow's own deletion within the revalidation window #270).
  • DagLoop subscribes to Moved: old owner evicts stale location entries fail-closed; the new owner re-forks drain so a wedged wake delivers immediately.
  • No-ALTER convergence migration fails closed (only rows with non-NULL workflow.directory AND a session with non-NULL directory); legacy NULL zombies stay NULL; idempotent (applied twice in probe C9).
  • R7-ext(a): two-writer directory-stamp whitelist encoded as an invariant update — wholesale revert of fdfc3fe stays 24/24 GREEN (pure contract addition).

Red-first evidence

M1 revert C1 → 22/2 RED on C4+R7-ext · M2 revert Moved subscription → 23/1 RED on C8 · M3 migration no-op → 23/1 RED on C9 · M4 always-claim → 23/1 RED on C7 · M5 drop spawn re-admission → 23/1 RED on C7. All restores byte-identical (sha256-verified).

Gates

Typecheck exit 0 · test/dag test/goal test/project 721 pass / 1 skip / 0 fail (58 files) · 24/24 location probes (21 baseline + C7/C8/C9, C4 flipped to pin resolution) · migration --check exit 0 · license-scope 5 pass · lint 4849/0 (budget 4850).

Review: design gate ACCEPT (combined model) + independent verify (mutation re-proofs) + standards/intent arbitration ACCEPT.

Known residual

Bounded re-stamp-vs-adoption race (tryClaimAdoption bumps time_updated with wall clock) — documented for follow-up. Directory-authority asymmetry (create stamps realpathed, Moved re-stamps not) documented at the claim site.

Closes #269, closes #270.

…onMoved (#269)

Root cause (#269): the execution-location key (WorkflowTable.directory) was
stamped at dag.create and never moved afterwards, while a SessionEvent.Moved
only rewrote SessionTable.directory. ownsSession conjuncts the directory over
ALL of a session's workflow rows, so after a move the rows disagree (the
pre-move workflows keep the old stamp) and the fail-closed ownership policy
leaves EVERY directory with no owner — the session's wakes wedge until restart.
Probe C4 pinned the wedge rather than fixing it (v1.0.13 decision).

Fix (design-gate Model a + b integration, atomic adoption):
- C1: the session projector's Moved projection now re-stamps every workflow row
  of the moved session to the payload-sourced destination directory, inside the
  SAME durable publish transaction as the SessionTable update — the stamp moves
  WITH the session in one transaction, so there is never a mixed-stamp window
  (no SessionTable read, no fs I/O in the projector).
- C5: DagLoop subscribes to SessionEvent.Moved. The instance whose directory no
  longer owns the moved session's workflows evicts its stale runtime entries
  (fail-closed), and the NEW owner re-forks the serialized wake drain so a wake
  that was wedged behind the old stamps delivers immediately (bounded time)
  rather than at the next idle event or restart.
- C6: no-ALTER idempotent convergence migration converges pre-fix divergent
  stamps to the session's current directory, preserving fail-closed NULL rows.

Probes (red-first; each was RED before the corresponding seam landed):
- C4 flips from pinning the wedge to pinning the resolution: exactly one live
  owner = the session's new durable directory, no cross-directory double
  adoption. RED evidence: wf1 stayed "/wtA" pre-C1.
- C8 pins the bounded-time resolution: a moved session's wedged wake reaches the
  new owner's admission seam immediately, delivered exactly once. RED evidence:
  the wake stayed wedged (poll timeout) pre-C5.
- C9 pins the migration convergence (fail-closed NULL preserved). RED evidence:
  a no-op migration leaves the stale stamp divergent.

Invariants preserved: fail-closed NULL, session-sourced create stamp, H1 latch,
R7 negative half. Only the write-once rule is narrowed to a two-writer whitelist
(create INSERT + Moved re-stamp) — encoded in the follow-up R7-ext(a) amendment.
Root cause (#270, shared territory with #269): admission was check-then-act —
an instance re-read the durable row (ownsWorkflow) and then published its
in-memory entry / materialized a child session. A deletion (Session.remove → FK
cascade) that commits between the passed ownership check and the action slipped
past it: the in-flight spawn could create a child for a workflow the cascade had
already removed. SpawnReady eviction was the mitigation, not a fence.

Fix (Model (b) conditional-claim fence of the accepted atomic-adoption design —
status+directory conditionality, no lease column):
- C2: DagStore.tryClaimAdoption — a single conditional UPDATE that matches the
  row only while it STILL EXISTS and is non-terminal, returning whether the
  admission claimed it. It converts the admission from check-then-act into
  act-with-condition; a committed deletion/terminal matches zero rows and the
  admission aborts atomically. Directory ownership is deliberately NOT re-done
  here: ownsWorkflow canonicalizes both sides and a SQL directory comparison
  would diverge (create stamps are realpathed, Moved re-stamps are not).
- C3: recoverWorkflow and the WorkflowStarted handler gate runtimes.set on the
  claim (after the ownsWorkflow directory revalidation), so a cascade in the
  final window aborts the adoption before it ever publishes an entry.
- C4: spawnNode re-admits atomically right before sessions.create — the fence
  that closes the spawn window nodeQueued's guard alone leaves open between its
  read and its publish. A committed deletion matches zero rows and the spawn
  aborts before any child session exists.

Probe C7 (red-first, park-gate style): parks the window-2 claim, commits a
deletion, releases, and asserts no post-deletion child survives.
RED evidence (fence conditionality reverted → always-claim): a child session is
created-then-cancelled for the deleted workflow, so `expect(cancelsA).toEqual([])`
fails (Received +3). Restoring the conditional claim makes the claim return false
on the deleted row and abort before sessions.create (GREEN).

All existing DAG-LOC-01 guards stay green (24 probes).
…ext(a) (#269)

Root cause (#269): the atomic-adoption redesign narrows the DAG-LOC-01
write-once directory invariant into a two-writer whitelist. R7-ext(a) previously
asserted "no UPDATE writes the directory column anywhere in the dag trees," which
held but left the new whitelisted writer (the Moved re-stamp) unpinned.

This is an invariant UPDATE, not a weakening — mandated by #269's own acceptance
("the directory stamp must move WITH the session in one transaction"):
- The ONLY sanctioned directory writers are the dag projector's WorkflowCreated
  INSERT (create-time stamp, onConflictDoNothing) and the session projector's
  SessionEvent.Moved re-stamp (payload-sourced, no SessionTable read).
- R7-ext(a) keeps its negative half (no .set({directory}) in the dag trees) and
  gains a positive whitelist half asserting the Moved re-stamp exists in the
  session projector (outside the dag trees, so R7/R7-ext stay scoped correctly),
  pinning the #269 resolution at the static-contract level.
- WorkflowTable.directory column comment documents the two-writer whitelist.

RED evidence (mutation reversion — Moved re-stamp removed from the session
projector): the whitelist expect fails, Expected true / Received false.

No user-visible semantic change; no runtime behavior altered in this commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant